Theory and Methodology A methodology for ®tting and validating metamodels in simulation

This paper proposes a methodology that replaces the usual ad hoc approach to metamodeling. This methodology considers validation of a metamodel with respect to both the underlying simulation model and the problem entity. It distinguishes between ®tting and validating a metamodel, and covers four types of goal: (i) understanding, (ii) prediction, (iii) optimization, and (iv) veri®cation and validation. The methodology consists of a metamodeling process with 10 steps. This process includes classic design of experiments (DOE) and measuring ®t through standard measures such as R-square and cross-validation statistics. The paper extends this DOE to stagewise DOE, and discusses several validation criteria, measures, and estimators. The methodology covers metamodels in general (including neural networks); it also gives a speci®c procedure for developing linear regression (including polynomial) metamodels for random simulation. Ó 2000 Elsevier Science B.V. All rights reserved.